Local resolution-limit-free Potts model for community detection 
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We report on an exceptionally accurate spin-glass-type Potts model for community detection. 
With a simple algorithm, we find that our approach is at least as accurate as the best currently 
available algorithms and robust to the effects of noise. It is also competitive with the best currently 
available algorithms in terms of speed and size of solvable systems. We find that the computational 
demand often exhibits superlinear scaling 0(L 1 ' 3 ) where L is the number of edges in the system, and 
we have applied the algorithm to synthetic systems as large as 40 x 10 6 nodes and over 1 x 10 9 edges. 
A previous stumbling block encountered by popular community detection methods is the so-called 
"resolution limit." Being a "local" measure of community structure, our Potts model is free from 
this resolution-limit effect, and it further remains a local measure on weighted and directed graphs. 
We also address the mitigation of resolution-limit effects for two other popular Potts models. 

PACS numbers: 89.75.Fb, 64.60.Cn, 89.65.-s 
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I. INTRODUCTION 

"Community detection" describes the problem of find- 
ing closely related sub-groups within a network. Appli- 
cations of the problem are extremely broad finding real- 
izations in the World Wide Web P, Q , food webs [3| , so- 
cial networks Q , protein interactions [|| , consumer pur- 
chasing patterns [|| , mobile phone networks @ , criminal 
networks Q , epidemiology Q , biological networks [H, @ , 
and other areas. A recent review of community detection 
appears in Ref. fioj ]. 

In this paper, we present an improvement to the Potts 
model as applied to community detection, and we demon- 
strate that it is extremely accurate, robust to noise, and 
competitive with the best available methods in terms of 
computational speed and the size of solvable systems. 
Our approach also corrects known problems encountered 
by other popular measures that do not properly resolve 
small communities when large sets of data are examined. 

The data in networks can often be cast as a graph con- 
sisting of members represented by nodes with pair-wise 
relationships between the nodes represented by edges. 
In general, these relationships can be specified in one 
direction along an edge, and they can be weighted or 
unweighted. The goal of community detection is to ef- 
ficiently separate clusters of closely related nodes from 
each other. Each cluster will have a proportionally higher 
number of internal edges compared its external connec- 
tions to each other community in the partition. 

The most popular quantitative community measure is 
that of "modularity" which was originally introduced by 
Newman and Girvan (Tlj . This measure constituted a 
work that transformed the field. Modularity measures 
the deviation of a proposed community structure com- 
pared to what is expected from an "average" case based 
on a particular random distribution (a "null model" ) . 

Our approach is a physics-inspired method that casts 
community detection as a Potts model spin glass. Com- 
munities correspond to Potts model spin states, and the 



associated system energy indicates the quality of a can- 
didate partition. Some earlier approaches utilizing Potts 
models are in [l2[ and Our particular model was 

originally inspired by a minimal cut method by Djid- 
jev 14J which is equivalent to modularity. The result- 
ing generalized Hamiltonian was previously presented 
by Reichardt and Bornholdt (RB) In their spe- 

cific implementation, RB generalized null model based 
approaches to community detection, including modular- 
ity as a special case, and elaborated on the connection 
between physics and community detection. Two other 
Potts model approaches are by Hastings [l6| and Ispola- 
tov et al. fl7j . 

Modularity and the RB Potts model (RBPM) utilize 
a random null model selected to evaluate the strength 
of a proposed community partition. Larger deviations 
(more intra-community links and fewer inter-community 
links) from the random case indicate better community 
structure. A null model is usually based on parameters 
of the graph being examined which allows the measure to 
"scale" to arbitrary graphs in an objective manner (see 
Sec. I VI Ap . Typical null models for the RBPM are: (i) an 
Erdos-Renyi null model (RBER) in which all edges are 
equally likely to be connected and (ii) the configuration 
null model (RBCM) in which edge connection probabili- 
ties are based on the current graph's degree distribution. 
For modularity, the dependence on the null model is in- 
herent to the definition of the measure. Within the RB 
scheme, the dependence on a null model is introduced by 
design. 

Fortunato and Barthelemy [l8[ later determined that 
modularity optimization can result in incorrect commu- 
nity divisions due to a resolution limit. The resolution 
limit is an inherent scaling in the expected number of 
communities q which roughly scales as y/~L where L is the 
total number of edges in the graph. The RBPM model 
is also subject to a resolution limit [l9[ due to how it is 
cast by design, analogous to modularity, in terms of an 
arbitrary null model comparison. The number of commu- 
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nities roughly scales as \f"f RB L, where r ) RB is a weight 
applied to the null model comparison. Optimizing cither 
measure (maximizing modularity or minimizing the Potts 
model energy) tends to merge small clusters in large sys- 
tems, or it may incorrectly partition large communities. 
Although the RBPM allows for an arbitrary choice of 
null model, the resolution limit was shown to persist fl9j 
regardless of the null model that is used. 

Our approach avoids a null model comparison [2(| . In- 
stead, it penalizes for missing edges directly in the en- 
ergy sum plj . In effect, a community is defined by its 
edge density as opposed to allowing each graph to inde- 
pendently define a community through the use of a null 
model. One consequence is that it removes the ability 
of the model to automatically scale the solution based 
on global properties of a graph (see Sec. I VI A|) . but the 
change results in a robust model with significant improve- 
ments to several desirable properties. 

In this paper, we demonstrate a simple but effective 
implementation of the g-state Potts model to commu- 
nity detection. In Sec. [H] we discuss our Potts model 
and some of its properties along with the RBPM and 
its main variants. We also explain the concept of the 
resolution of a partition. In Sec. IIII1 we present our al- 
gorithm, and Sec. IIVI illustrates its accuracy compared 
to several other approaches. Our Potts model and the 
RBCM model are directly compared in Sec. [Vj Issues 
regarding local and global measures and the resolution 
limit for general graphs are addressed in Sec. IVII We 
solve two examples in Scc. lVIII and conclude in Sec. I Villi 
Appendix A argues that the unweighted variant of the 
RBER model can be strengthened to eliminate the reso- 
lution limit. In Appendix B, we explain the variation of 
information (VI) metric which we use in Sec. [V] Appen- 
dices C and D elaborate on details related to the bench- 
mark in Sec. IV Bl Appendix E explains a community 
detection transition observed in the same benchmark. 



II. POTTS MODEL HAMILTONIANS 

Quality functions in community detection evaluate the 
"best" partitions based on at least two criteria: Edges in- 
side a community strengthen the community. In order to 
consistently avoid a trivial solution (a single community) , 
a quality function must also apply a "penalty function." 
The most common method compares community degree 
distributions to an "expected" distribution based on a 
null model, but other approaches are possible. 



A. Absolute Potts model 

Our Potts model directly penalizes for missing edges 
within a community. The result is a robust model that 
is highly accurate, a local model for general graphs 
(weighted, unweighted, and directed), and free of the 
resolution limit. We also connect the introduced model 



weight 7 to the resolution of a system and relate the 
interaction energies to the stability of communities. 



1. Hamiltonian 

We construct the Potts model with the following con- 
siderations. Edges inside communities and missing edges 
outside communities are both favorable for a well-defined 
community structure, so the energy of the system is low- 
ered by these arrangements. The opposite holds for edges 
outside communities and missing edges inside communi- 
ties. This generalized Potts Hamiltonian is [ll| 

where {A^} is the set of adjacency matrix elements: 
Aij = 1 if nodes i and j are connected and is if they 
are unconnected, and Jij = (1 — Aij). The edge weights 
({a^} and {bij}) and connection matrices ({A^} and 
{Jij}) are defined by the system. The Potts spin vari- 
able <ii takes an integer value in the range 1 < c.; < q 
which designates the community membership of node i 
(node i is in community k if o~i = k). The number of 
communities q can be set as a constraint, or it can be 
determined from the lowest energy configuration. The 
Kroneker delta 5(ui, Uj) = 1 if o~i = o~j and if Oi =/= Uj. 

The spin glass type Potts model of Eq. (TTJ) can be re- 
duced, up to an additive constant, to a form that greatly 
simplifies implementation. For comparison, we introduce 
a form similar in appearance to the notation used by RB 

^(W) = — 2 ( a «^« ~ lh y hj)o~{o- u o-j). (2) 

Spins interact only with other spins in the same commu- 
nity (cr, = o~j). The introduced factor of 7 allows the 
model to scale the relative strengths of the connected 
and missing edge weig hts. The generality of the weights 



{{ ai j} and {bij}) [2C 



enables the study of directed 



graphs, weighted graphs, and graphs with missing link 
weights (i.e., levels of "repulsion"). Traag and Brugge- 



231 ] also presented a generalization of the RBCM 
that similarly allows for "negative" link weights. Un- 
weighted graphs use edge weights of a^ = bij = 1. 

The Hamiltonian of Eq. ([2]) describes a system wherein 
spins in the same community interact ferromagnetically if 
they are connected and antiferromagnetically if they are 
not connected. We identify communities by minimizing 
Eq. ([2]), and despite a global energy sum, our model is a 
local measure of community structure (see Sec. IVT|) . We 
refer to Eq. @ as an "absolute" Potts model (APM) as 
it is not defined relative to a null model. Although our 
analysis in this paper will focus on the static APM, it 
is defined for both static systems and dynamic networks 
with time-dependent weights and adjacency matrices. 
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2. Resolution 

Intuitively, the resolution of a community partition is 
set. on average, by the strength of intra-community con- 
nections. That is, the resolution of the partition may be 
specified by the typical edge density of the communities 
within the partition. Communities with substantially dif- 
ferent edge densities have different qualitative features. 

In social networks for example, a partition intending to 
convey the "close friends" within a network would intu- 
itively have a higher typical edge density than a partition 
that includes all "acquaintances" since the disparate ac- 
quaintances are much less likely to know each other. Ide- 
ally, a partition should contain communities that con- 
vey similar qualitative information (i.e., similar "levels" 
of association). In practice, it will contain communities 
with different edge densities, but intuitively the differ- 
ences would not be drastic for a given resolution. 

For unweighted graphs, the edge density p s of commu- 
nity s is p s = £ s /£™ ax where l s is the number of edges 
in the community. ^ nax = n s (n s — l)/2 where n s is the 
number of nodes. The model weight 7 in Eq. is re- 
lated to the minimum edge density of each community, 

Pmin > — Tj (3) 

7+1 

which is determined by calculating the minimum com- 
munity density that gives an energy of zero or less. Al- 
ternately, we can use an inductive argument based on the 
maximum intercommunity edge density that causes two 
arbitrary communities to merge. For weighted graphs, 
we define a "weight density" p s = w s /w™ ax where w s 
is the sum of all weighted edges in community s and 
the "maximum weight" iu™ ax = uJ s ^" lax where w s is the 
average edge weight. The minimum density is p min > 
j/^ + Ws/lls) where u s is the average weight of the miss- 
ing links. Without 7, the model is restricted to solving 
one particular resolution of a system. This relation be- 
tween 7 and the community density is distinctly different 
from a resolution limit because the communities are de- 
termined through only local constraints (see Sec. IVI|) . 

3. Community and node stability 

From Eq. ([2]), the interaction energy E rs between com- 
munities r and s is 

E rs = -w rs + ju rs (4) 

where w rs is the energy sum over all edges and u rs is 
the energy sum over all missing links strictly between 
the two communities. E ss = E s is the internal energy of 
community s where the energy sum is over all internal 
edges and missing links. When E s ~ 0, the assignment 
of community s is more sensitive to local perturbations. 

Similarly, the interaction energy E r i of node i with 
community r is given by Eq. (jj). If E S i—E r i ~ for node 



i in community s, then the node is susceptible to displace- 
ment by system perturbations. When a node contributes 
a large fraction of the energy E s of its own community, 
the community is susceptible to disruption if the node is 
moved. Equation ((4]) indicates the strong local behavior 
of the APM (see Sec. IVIj) . For general graphs, the inter- 
action energy of node i or community s is measured only 
by its own edges or missing links with each community. 

B. RB Potts models 

We compare the APM to the RBPM in order to demon- 
strate improvements in accuracy and locality despite the 
apparent similarity in the models. The RBPM, using an 
arbitrary null model, is defined as [Til ] 

Urb({<t}) = -- ( A *j ~ ~f RB Pij) 6(o-i,o-j) (5) 

i¥=3 

where we include the overcounting scale factor of 1/2. 
The term pij is the probability that nodes i and j are 
connected, and it incorporates the dependence on the 
arbitrary null model. 7 HB is the weight applied to the 
null model. The most frequently used null models are an 
Erdos-Renyi null model and the configuration null model 
(see Sec.|I]). For later reference, they are explicitly given 
by p^ = p for the Erdos-Renyi null model 

?OW) = ~\ £ iAii ~ InuP) Sfa, Vj) (6) 

and by pij = kikj/(2L) for the configuration null model 

where ki is the degree of node i. Equation ([7]) appears to 
be the more preferred model since the configuration null 
model incorporates information about the degree distri- 
bution of the graph under consideration. 

When j RB = 1, the RBCM of Eq. © is equivalent 
to modularity [l5| up to a scale factor of —1/L, The 
APM can be made eq uivalent to the RBER model for 
unweighted graphs [2J] (see also Appendix A). We ad- 
dress weighted generalizations of both models and their 
effect on model locality in Sec. IVI El Despite the similar 
forms of the Hamiltonians of Eqs. © and ([5]), the model 
weights 7 and j RB perform distinctly different roles in the 
two models. In the APM, 7 directly adjusts the weight 
applied to missing edges. In the RBCM, j RB adjusts the 
weight applied to the null model. We contrast the accu- 
racy of the APM and the RBCM in Sec. El 

III. ALGORITHM 

Our algorithm moves nodes by identifying which com- 
munity they may be moved into so that the system energy 
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is lowered. The algorithm proceeds until no more node 
moves are possible. This "orthogonal steepest descent" 
algorithm (selecting the path of steepest descent for only 
one spin Oi at a time) is extremely fast. We introduced 
our initial implementation of the algorithm in j20| . A 
summary of the efficiency of several algorithms appears 
in l25ll. A number of algorithms were compared in (26j 
and [27] where algorithms similar to ours performed very 
well when optimizing modularity. Combined with the 
APM, it is exceptionally accurate. The steps of the al- 
gorithm are: 

(1) Initialize the system. Initialize the connection ma- 
trices (Aij and Jy) and edge weights (oy and bij). The 
system begins in a "symmetric" state wherein each node 
forms its own individual community (qo = N). If the 
number of communities q is constrained (e.g., Figs. [T] 
and [H]) , we randomly initialize the system into qo = q 
communities. 

(2) Optimize the node memberships. Sequentially "pick 
up" each node and scan its neighbor list. Calculate the 
energy change as if it were moved to each connected clus- 
ter. Immediately place it in the community with the 
lowest energy (optionally allowing zero energy changes). 
Each iteration through all nodes is O(L). 

(3) Iterate until convergence. Repeat step (2) until an 
energy minimum is reached where no node moves will 
further lower the system energy. 

(4) Test for a local energy minimum. Manually merge 
any connected communities if the merge(s) will further 
lower the energy of the system. If any merges are found, 
return to step (2) for any additional node-level refine- 
ments. We estimate that the computational cost is 
O(Llogq) which is generally smaller than the node opti- 
mization cost in steps (2) and (3). 

(5) Repeat for several trials. Repeat steps (1) - (4) for 
t independent "trials" and select the lowest energy result 
as the best solution. By a trial, we refer to a copy of the 
network in which the initial system is randomized. 

The symmetric initialization for the nodes in step (1) 
is not uncommon in the literature [f| [28l - [30j . Steps (2) 
and (3) are the fundamental elements of the algorithm 
which arc similar to portions of algorithms used elsewhere 
@, [29[. The number of iterations is generally O(10) for 
large systems, but it can be higher for "hard" problems. 
In step (4), the community merge test is sometimes nec- 
essary because certain configurations, particularly heav- 
ily weighted graphs with 7 <C 1, more easily trap the 
node-level refinements [steps (2) and (3)] in local energy 
minima. The merge test is not generally a major concern 
for 7 > 1. 

The order of node moves is significant, so the addi- 
tional trials in step (5) sample different regions of the 
energy landscape and can yield different solutions even 
with the symmetric initialization in step (1). We opti- 
mize solutions by increasing the number of trials where 
the greatest benefit occurs for problems of "intermedi- 
ate" difficulty (e.g., see the data for the APM in Fig.rjJ). 
The number of trials t is generally 0(10) or less. 
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FIG. 1: (Color online) Plot of the percentage of corr ectly 
identified nodes p versus the average external degree kout [32I ]. 
The average node degree is k = 16. The data for the APM 
of Eq. I© and the RBCM of Eq. both use 7 = -y RB = 1. 
Both models use the algorithm in Sec. IIHI with q = 4 by 
constraint (see text regarding the RBCM/Symmetric data). 
The APM is at least as accurate as SA (error bars are for t = 
10 optimization trials), and the RBCM performs excellently 
also. Each point is an average over 500 runs. 



Empirically, the overall solution cost often scales as 
0(tL 13 logk) where k is the average node degree. The 
factor of \ogk applies for large sparse matrix systems. 
The algorithm can accurately scale to at least O(10 7 ) 
nodes and O(10 9 ) edges with a calculation time of several 
hours [H (see Sec. EH]). 



IV. ACCURACY COMPARED TO OTHER 
ALGORITHMS 

We test the accuracy of our method compared to sev- 
eral other algorithms using a common benchmark [32| . 
The benchmark is very small by current standards with 
an unrealistically symmetric community structure, but 
its frequent use provides a means of comparing the ac- 
curacy of various algorithms that have been presented in 
the literature over time. The problem defines a system 
of N = 128 nodes in q = 4 clusters of n = 32 nodes 
each. Each node is assigned an average of k = 16 edges 
of which ki n are randomly assigned inside its own com- 
munity. k out edges are randomly assigned to nodes in 
other communities such that k = ki n + k out . We then 
attempt to verify the defined community structure. 

In Fig. [TJ we plot the "percentage" of correctly identi- 
fied nodes p as a function of k out . For consistency with 
other data in Fig. [T] we use the same measure of percent- 
age accuracy as Newman [32j|. We use q = 4 communi- 
ties by constraint and test several levels of optimization 
(i = 5, 10, and 40). At t = 10, our method maintains an 
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FIG. 2: (Color online) The figure depicts a simulated three- 
level heterogeneously-sized hierarchy with N — 256 nodes 
[22l I35I ]. The innermost level 3 has 53 = 16 communities 
with a randomly assigned average density of p 3 = 0.90. The 
intermediate level 2 has 32 = 5 communities and an aver- 
age density of p 2 = 0.47 that is constructed by connecting 
the constituent level 3 sub-groups at an intercommunity edge 
density of P2 = 0.3. Level 1 is the completely merged system 
with an average density of p l = 0.18, and it is constructed 
by connecting nodes in different level 2 communities with an 
intercommunity edge density of pi = 0.1. 

accuracy rate of 95% or better up to kout = 7.5. 

Several sets of data were assimilated by Boccaletti et 
al. (33j where the most accurate algorithm was simulated 
annealing (SA) although it is computationally demand- 
ing [25| . Other accurate data are by Hastings [l6j , Gud- 
kov et al. [3fjj . and our algorithm in Sec. IIIII applied to 
the RBCM with j RB = 1 (modularity) and t = 10. Our 
algorithm is as accurate as SA when used with the APM. 

The APM, one set of our data for the RBCM, and the 
data by Hastings impose q = 4 as a constraint; so us- 
ing a constrained q may affect the accuracy rate in this 
problem. The initial state of the system substantially 
influences the accuracy of our algorithm for the RBCM 
when q is unconstrained and when starting from an initial 
state of one node per cluster (symmetric) or a random 
state (not depicted) HH . A recent analysis [I?} showed 
our multiresolution algorithm (22j applied to this bench- 
mark using the APM with unconstrained q where it was 
also very accurate, among the best of tested algorithms. 

V. ACCURACY COMPARISON OF POTTS 
MODELS 

We compare the APM of Eq. © to the RBCM of Eq. 
(O with two test systems. First, we solve for the different 
levels of the synthetic hierarchy depicted in Fig. [5] with 
the results given in Fig. [3] Second, we create a set of 
strongly defined systems with high community edge den- 
sities and increasing levels of noise. A sample graph is 
depicted in Fig. 2] with the results given in Figs. [5] and |U 




FIG. 3: (Color online) Plot of VI V vs model weights 7 or 
7 HB for the APM of Eq. © and the RBCM (configuration null 
model) of Eq. 0, respectively. The plots illustrate how the 
model weights operate in the respective models. We use the 
algorithm in Sec. IIIII for both models to identify the hierarchy 
depicted in Fig. [2] using t = 1 and 4 trials. We calculate VI 
with respect to level 2 of the hierarchy in panel (a) and level 
3 in panel (b) . Both models exactly identify both levels of the 
hierarchy at t = 4. The APM perfectly identifies both levels 
at t = 1 which is slightly better on average than the RBCM, 
and it has a more stable solution for level 3. Each point is an 
average over 100 solutions. 

The APM proves to be very robust to noise in the system. 
We use the VI information metric V (see Appendix B) to 
compare solved partitions with the constructed networks. 



A. Three-level hierarchy 

We identify two levels of a constructed hierarchy [22|, 
[H| using both the APM and RBCM models. The three- 
level hierarchy is depicted in Fig. [5J and the results are 
given in Fig. [31 The system has N = 256 nodes divided 
into <73 = 16 communities at level 3 with sizes as noted in 
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Fig. [2] Edges in each community are randomly assigned 
with a probability of p 3 — 0.90. These communities are 
grouped as shown into q2 — 5 communities that define 
level 2 of the hierarchy. The average internal density of 
level 2 communities is p 2 = 0.47 which are defined by 
randomly connecting nodes in the respective sub-groups 
of level 3 at an intercommunity edge density of p 2 = 0.3. 
Level 1 is the completely merged system which is defined 
by randomly connecting nodes in sub-groups of level 2 at 
an intercommunity edge density of pi = 0.1. 

We apply the algorithm of Sec. IHII to both models and 
solve a large range of model weights 7 or j RB , respec- 
tively, in order to illustrate the differences in the two 
models. In Fig. [31 we plot VI V as a function 7 or r y RB 
[111 using t = 1 and 4 trials. VI is calculated between 
the respective solutions and the level 2 or 3 partitions of 
the hierarchy. These data are then plotted in panels (a) 
and (b), respectively. Both models exactly identify both 
levels of the hierarchy at t = 4. The APM is slightly 
better in accurately identifying them with t = 1, and it 
has a more "stable" solution in panel (b). 



B. Noise test 
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FIG. 4: (Color online) A sample graph with N = 512 nodes 
for the noise test in Sec. IV Bl In this sample, the node de- 
grees are initially defined in a power-law distribution with 
an average {k) a = 5.4, maximum fc max = 100, and exponent 
a = —2. Communities have a power-law distribution of sizes 
with a minimum n m i n = 4, maximum n max = 50, and expo- 
nent P = — 1. These communities are then strongly defined 
by connecting all internal community edges (pi n = 1). 



The concept of "noise" in community detection corre- 
sponds to "extra" edges that connect a node to commu- 
nities other than its best assignment (s). In general, we 
cannot initially distinguish between edges contributing to 
noise and those constituting edges within communities of 
the best partition(s). Community detection methods ex- 
perience the effects of noise in at least two distinct ways: 
(1) The edges due to noise act to obscure the best parti- 
tion^) in an algorithm by creating "confusion" for early 
community assignments (a dynamical effect). (2) The 
extra edges influence the quantitative evaluation of the 
best community assignments (a "metric" effect). In some 
models, this second effect can negatively impact the con- 
tribution of edges that comprise the best communities. 



1. Benchmark 

We test the accuracy of APM and RBCM models with 
high levels of noise in a series of strongly defined systems 
with "realistic" distributions of community sizes. Specif- 
ically, we define a set of communities with a power-law 
distribution of community sizes specified by an exponent 
/3, minimum size n m j n , and maximum size n max . We add 
random edges to the system, largely defining the inter- 
community noise, based on a power-law distribution of 
node degrees given by an exponent a, average power-law 
degree (k) a (or minimum degree £; m ; n ), and maximum 
degree £; max [371 ] . This initial framework is similar to a 
benchmark by Lancichinetti et al. [2?], H3- We then 
connect internal community edges at a high density pi n . 

The strongly defined communities provide unambigu- 
ous partitions where the large level of noise does not 



significantly alter the optimal solutions (see Sec. IV BH) . 
This density-based definition of community structure is 
consistent with concepts proposed for community identi- 
fication by Palla et al. Q . We solve for the systems using 
the algorithm in Sec. Mil for both models with t = 1 and 
4 trials and using SA for the RBCM with t = 4. 



2. Accuracy results 

Figures 2] and O show a sample system and the first 
test results, respectively. We use two system sizes of 
V aj b = 512 and N c ^ = 4096 nodes, respectively. The 
initial power-law degree distribution uses a = —2; and 
the maximum degree constraints are k™^ K = 100 and 
t;™ = 1000, respectively. We increment the average 
power-law degree (k) a to vary the system noise (the aver- 
age external degree k out ~ (k) a for large systems). Com- 
munity sizes range from n m i n = 4 to n max = 50 nodes and 
are distributed according to /3 ajC = — 1 or /?b,d = —2, re- 
spectively The internal community edges are maximally 
connected at a density of pi„ = 1 . 

We plot VI V versus k out for both Potts models where 
VI is calculated between the solved partition and the 
generated graph (V a m b ax = 9 and V^* = 12). For the 
APM, we use 7=1 for every solution, and we allow 
zero energy moves after the system reaches an initially 
converged state. For the RBCM, we subjectively select 
the best solution corresponding to the highest accuracy 
j RB independently determined for each k ou t given the 
known answer (see Appendix C). We further solve the 
system via SA at this best value of "f RB for comparison. 
We average over 100 graphs for each point in panels (a) 
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FIG. 5: (Color online) Plot of VI V between solved and known 
test systems in Sec. IV Bl as a function of the average exter- 
nal node degree k ou t. The system sizes are -/V a ,b = 512 in 
panels (a) and (b) (with a sample system depicted in Fig. Q 
and iV c> a = 4096 nodes in panels (c) and (d). The graphs 
are solved with the APM and the RBCM using the algorithm 
in Sec. Mil with t = 1 and 4 trials. We use 7 = 1 for the 
APM on all solutions, and we subjectively select the best -y RB 
for the RBCM independently for each k ou t (see Appendix C). 
For comparison, we also solve the system at this best 7 BB us- 
ing SA. System noise is randomly assigned in an approximate 
power-law degree distribution [331 with an exponent a = — 2, 
an average degree {k) a , and maximum degrees of k™* — 100 
or fc™^ x = 1000, respectively. Constructed communities are 
randomly assigned in a power-law size distribution [3j| spec- 
ified by an exponent /3 a , c = — 1 or /3b, d = — 2, minimum size 
"■min = 4, and maximum size n ma x = 50. Communities are 
then maximally connected with pi n — 1. Even with t = 1, 
the APM is almost perfectly accurate for most tested param- 
eters in this problem. See Appendix E regarding the accuracy 
transitions in panel (d). Data are averaged over 100 graphs 
in panels (a) and (b) and 25 graphs in panels (c) and (d). 



and (b) and 25 graphs in panels (c) and (d). 

In panels (a) and (c) , the advantage in accuracy for the 
APM is modest. The accuracy of the RBCM increases 
in panels (a) and (b) at higher levels of noise due in 
part to the fact that the degree distribution is becoming 
more uniform as we increase (k) a but keep fc max constant. 
While the RBCM performs excellently in many cases, the 
APM outperforms it to varying degrees for most tested 
parameters and levels of noise. Moreover, the APM is 
often able to almost perfectly solve the system. 

The rapid increases in VI for both models in Fig. Efd) 
are due to transition effects described in Appendix E. We 
subjectively select the best 7 KB in this paper, but note 
also that our algorithm can slightly outperform SA in 
accuracy in many cases for either Potts model (see Sec. 
IV B 31 and Appendix D). See Sec. I VI Dl regarding how the 
high levels of noise in this test actually mitigate the effect 
of the resolution limit for the RBCM. 



FIG. 6: (Color online) Plot of VI V between solved and known 
test systems in Sec. IV Bl vs the average external node degree 
kout- In panels (a) through (d), system sizes are iV a> b = 512 
and iV c ,d = 4096 nodes, respectively. The graphs are solved 
with the APM and the RBCM using the algorithm in Sec. [Ill] 
with t = 4 trials. The constructed configurations are identi- 
cal to those used in Fig. [5] In this plot, we test two different 
initial states for the solutions: a symmetric initial state and 
a random power-law distribution (see text). We use 7=1 
for the APM on all solutions, and we subjectively choose the 
best 7 HB for the RBCM for each kout ( see Appendix C). For 
comparison, the results for SA with t = 4 are also depicted 
and are solved using this best value of r y RB . The APM and S A 
with the RBCM show no difference in accuracy between the 
symmetric and random initial states. A symmetric state ap- 
pears to be the favored starting configuration for the RBCM 
when using a greedy algorithm in this benchmark. In fact, this 
symmetric initial state allows the RBCM to slightly outper- 
form SA in accuracy (see text). We average over 100 graphs 
in panels (a) and (b) and 25 graphs in panels (c) and (d). 



3. Dependence on initial condition and SA accuracy 

A community detection algorithm should ideally be ro- 
bust with respect to the initial state that is used to solve 
the system. We show that APM displays this feature, 
and we contrast the result with the RBCM when using 
the greedy algorithm in Sec. lIIII We further elaborate on 
the accuracy of SA compared to this greedy algorithm. 

In Fig. [SI we plot VI V vs k ou t where increasing k out 
corresponds to higher levels of system noise. We measure 
V between the solved and constructed systems where the 
defined systems are identical to the previous subsection. 
We test both models beginning from two different initial 
states: a symmetric initial state of one node per cluster 
with cjq = N and a random power-law configuration with 
go — q which is different than the defined answer. For 
simplicity, this random initial state uses the same distri- 
bution parameters (/3 a ,c = — 1 or /?b,d = —2, n m ; n = 4, 
and n max = 50) that are used to generate the answers. 
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The best solutions for the APM arc robust to the initial 
state of the system in this benchmark, including during 
the major accuracy transition in panel (d), despite using 
a greedy algorithm. The symmetric initial state performs 
very well for the RBCM and is the favored starting con- 
figuration compared to the random power-law state. The 
situation is reversed for the RBCM on the benchmark 
in Fig. Q] in Sec. IIVI where the symmetric initialization 
performs worse than a random initial state with go = 4 
communities (with or without constraining q in the dy- 
namics [HI) although that benchmark has an unrealisti- 
cally symmetric community structure. While optimizing 
the RBCM often provides excellent partitions, this differ- 
ence in accuracy between initial states indicates that it is 
more easily trapped in unfavorable regions of the energy 
landscape than the APM when using a greedy algorithm. 

As expected, SA with the RBCM shows no difference 
in accuracy for either initial state, but the greedy algo- 
rithm outperforms SA in terms of accuracy when using 
a symmetric initial state (see also Appendix D). This re- 
duced accuracy for SA compared to a greedy algorithm 
is not isolated to this benchmark. Lancichinctti and For- 
tunate p7j compared the accuracy of severa l alg orithms 
using their benchmark [38| . One result in [27j showed 
that a similar greedy algorithm optimizing modularity 
[equivalent to 7 BB = 1 in Eq. ([7])] by Blondel et al. [6[ 
also outperformed SA in accuracy on that benchmark. 
See also Good et al. dH regarding difficulties associated 
with modularity optimization in practical problems. 



4- Noise tolerance discussion 

Even at low levels of noise, these benchmark graphs ex- 
ceed the proposed definition of so-called "weak" commu- 
nities [40|, but the communities are not ill-defined from 
an intuitive standpoint within the tested range of noise. 
In panel (d) for example, at k ou t — 370 the average num- 
ber of edges connecting a given node to another com- 
munity is I ~ 1 because the k out edges are randomly 
spread over (q^ — 1) ~ 370 communities. This value is 
small compared to the average internal degree fc.;„ ~ 10 
and the average number of missing links with an external 
community ( (m) — £) ~ 10 where (m) is the average com- 
munity size. Thus, the communities remain well-defined 
particularly given their high edge density p, n = 1. 

The two Potts models respond to the noise in the sys- 
tem in distinctly different ways in terms of how the com- 
munity measure is calculated. Noise complicates com- 
munity assignment decisions for the RBCM because the 
configuration null model [the second term in Eq. (J7J] in- 
corporates the contribution of all edges, including noise, 
for every node assignment evaluation even after a reli- 
able solution "kernel" is located during early stages of 
the solution dynamics (the metric effect of noise) . 

The APM evaluates all edges for community assign- 
ment decisions through relative energy calculations, but 
Figs. [S] and |5] demonstrate that the best solution is often 



completely unaffected by the system noise if the algo- 
rithm can navigate sufficiently close to the solution. Once 
an initial solution kernel evolves during the early stages 
of the algorithm dynamics, confusion caused by random 
system noise is often easily mitigated by the missing edge 
energy penalty [the second term in Eq. @]. That is, the 
metric effect of noise on the APM is very favorable so 
that the main challenge caused by noise in the network 
is often due to the dynamical effect of noise (early incor- 
rect assignments) that affects both models. 

We could further improve the accuracy of the APM 
using a more robust, but much slower, algorithm such 
as SA. Nevertheless, this benchmark illustrates that the 
energy landscape of the APM is more easily navigated, 
particularly for 7 = 1, than the RBCM. The energy land- 
scape of the APM is more difficult to navigate for 7 <C 1 
as compared to 7 > 1 or when communities are not as 
strongly defined as they are in this test, but the model 
maintains an exceptional accuracy [22l |27| . 

VI. RESOLUTION LIMIT 

The quantitative approaches of modularity [ll[ and the 
RBPM [TH, [HI in Sec. Ill B I were implemented by incor- 
porating global graph parameters into the models. Both 
models marked important progress in the field of com- 
munity detection, but Rcfs. [ll| and [l9[ noted an un- 
intended consequence of using global community mea- 
sures — an imposed resolution limit. The resolution 
limit restricts the solutions of affected models so that 
they cannot correctly resolve all communities of a sys- 
tem in certain non-pathological cases. For modularity 
and the RBCM, the number of communities in a system 
tends toward \[~L [l~8[ and y/j RB L [lj[ , respectively. The 
models have difficulty properly resolving small commu- 
nities in large systems and may incorrectly divide large 
communities. 

We first discuss local versus global measures, and we 
then illustrate the resolution limit for the RB Potts mod- 
els, including modularity as a special case. We also show 
that the APM is free of resolution-limit effects because 
it is a local measure of community structure. 

A. Local vs global measures 

Including global dependences in quantitative commu- 
nity detection models was apparently rooted in the need 
to objectively determine the community structure of ar- 
bitrary graphs. The assumption is that the global proper- 
ties of the graph should imply its local community struc- 
ture. Global dependences make the partition solution 
objective since it allows the same quantitative model to 
automatically rescale to any graph, but they also became 
the central element that caused a resolution limit. 

One suggested solution [l8|, [lj| to the resolution limit 
is to define a local measure of community structure. That 
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is, community evaluations are made based only on local 
features of the graph in the neighborhood of the involved 
nodes and communities. Some approaches that provide 
local community detection methods include clique per- 
colation 0, EIL analyzing random walk s I42L l43j . "la- 
bel propagation" [29| (and one variant in 144]), and local 
variants of modularity [f| EH- See Sec. IVI E 41 and Ap- 
pendix A regarding the RBER model. The APM is also 
a strongly local measure of community structure. 

Local models possess beneficial properties for solving 
some networks such as: large networks that are "defined" 
as the network is explored (e.g., the World Wide Web), 
incompletely known networks (e.g., social interactions), 
coarse partitioning and refinement algorithms, and dy- 
namic networks. Communities sufficiently isolated from 
graph changes do not have to be repetitively updated as 
the network is modified. There exists work for modular- 
ity [46| that preserves the measure as the system is scaled, 
but a local model does not require this extra buttress- 
ing. Further, several of the most accurate community 
detection methods [27], EH are based on essentially local 
methods or models of community detection [22, E2, EH . 

However, using a local measure of community struc- 
ture returns to the subjectivity problem. How does the 
model objectively determine the community structure of 
arbitrary graphs? Stated specifically for the APM, how 
does one choose the "correct" resolution(s) [i.e., value(s) 
of 7 in Eq. ([2])] that will best solve the system? Sev- 
eral answers to this problem are as follows although the 
concepts are not restricted to local models. 

One approach is to define a community independent of 
the graph being solved. For example, we might seek to 
identify all communities of "close friends" in a social net- 
work regardless of the size of the graph. For the APM, 
Eq. ([3]) relates the model weight 7 to the minimum com- 
munity edge density for all communities in a partition. 

Some other methods, which are beyond the scope of 
this paper, define an algorithm or measure that can de- 
termine which resolutions (see Sec. Ill A2[) are the best 
partitions for the network. Arenas et al. [4?} varied 
a weight parameter with modularity and tracked stable 
partitions. Kumpula and co-workers [H, EH as wen as 
Fenn et al. [5(| also explored stability approaches for 
the RBCM. Our multiresolution method [22( utilized in- 
formation comparisons among independent solutions to 
quantitatively evaluate the best resolutions. Zhang et al. 
[51| used a topological weighting strategy. Cheng and 
Shen [43[ used the stability of random walker diffusion 
dynamics to identify the most relevant resolutions. 




FIG. 7: (Color online) Each graph is a circle of cliques with 
N total nodes and L total edges, (a) A set of q cliques with 
m nodes each are connected in a circle by q links, (b) r 
consecutive cliques are each grouped together. Intuitively, one 
would expect that any measure should resist merging these 
communities on any system scale (e.g., N, L, or q) if m > 3. 



each clique is a separate community. The total number 
of links is L and the number of nodes in the system is N. 
The total number of links between the cliques is q. The 
number of nodes in each clique m can be varied indepen- 
dently of q. From Eq. @, the APM energy is 



I 

#a = --qm(m - 1). 



(8) 



This energy E a has no finite extremum with respect to 
any global parameters of the graph. The analogue to 
Eq. ([8]) for modularity and the RBCM is where the res- 
olution limit was demonstrated. That is, those models 
have minima, q* mod = \fh and q* RB = ^7 HB £, respec- 
tively, in the expected number of communities. Neither 
of these values correspond to the intuitive partition (q 
clique communities) for all system sizes. 

Figure [7Jb) depicts sets of r cliques grouped together. 
The specific conditions, based on 7 flB , for r neighboring 
cliques to merge arc given by the following relations. The 
RBCM of Eq. ([7]), using the configuration null model, 
includes modularity as a special case when j RB = 1 . Two 
neighboring cliques (r = 2) [l9j will merge if 



Irb < 



q 



m{m - 1) + 2 



(9) 



The dependence on the number of cliques q is a problem 
since this condition for r y RB can always be satisfied if q is 
large enough (see also Sec. IVIDj) . For example, if m = 3 
and 7 BB = 1, the cliques merge if q > 8. 

When using the RBER model with an Erdos-Renyi null 
model in Eq. ((5]), neighboring cliques merge if 



B. Circle of cliques 



< 



q — 1 jm 
m(m — 1) + 2 



(10) 



Fortunato and Barthelemy jl8j and Kumpula et al. 
[l9| identified a resolution limit in the respective mod- 
els in part by considering the unweighted system shown 
in Fig. [7J a set of q cliques (maximally connected commu- 
nities) connected in a circle by single edges. In Fig. [7Ja), 



We can always choose q large enough to induce a merger 
of neighboring cliques for any j RB (see also Appendix A) . 
These results generalize so that a resolution limit can be 
found to apply for an arbitrary choice of null model fl9T ] 
when using the RBPM of Eq. ©. The APM energy E h 
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The RBER model yields a merge condition of 



FIG. 8: (Color online) A large graph G has N nodes and 
three sub-divisions depicted, one potentially large sub-graph 
G' with N' nodes and two distinct communities A and B with 
n and m nodes, respectively. In panel (a), G is unweighted, 
and communities A and B are joined by I edges. In panel (b), 
G is a weighted graph. For visualization purposes, solid lines 
depict weighted edges, dashed lines depict weighted missing 
links, and the link thickness depicts a relative link weight. 
Communities A and B are joined by weighted edges with a 
summed weight of w. Weighted missing links have a summed 
weight of u. All other graph features are left unspecified but 
are consistent with the community designations. 



of the configuration in Fig. [7(b) with r merged cliques is 
7+1 



E h = — 



1 - 



-qm(m — 1) 

7 rm — 1 
7+1 m — 1 



We compare the energies in Eqs. 
that r — 2 cliques merge if 

1 



7 < 



1 



^-M • (id 

rm{m — 1) 
J and (HI]) and find 

(12) 



This merge condition depends only on the local variable 
to. Solving the system with 7 = 1 will ensure that neigh- 
boring cliques will not merge on any global scale (e.g., 
N, L, or q) of the system. Since 7 adjusts the weight 
applied to missing links, we can force a merger of neigh- 
boring cliques if we reduce 7 to a sufficiently low value. 
At to = 3 for example, we can force a merger if 7 < 1/8. 



C. Heterogeneous communities 

Resolution-limit effects can be exacerbated when com- 
munities of substantially different sizes are present. 
Danon et al. addressed improvements for modularity 
to better resolve heterogeneous structures [l| with New- 
man's algorithm in [32j . The APM deals with heteroge- 
neous communities naturally. 

Figure [5(a) depicts a large graph G with three di- 
visions. Communities A and B have n and to nodes, 
respectively, and are connected by I edges. Sub-graph 
G has N' nodes with an unspecified structure. For the 
RBPM of Eq. ([5]), using a generic null model, the number 
of edges I that causes communities A and B to merge is 
of the order [H 



> 



N 



(13) 



I > 



2L 



N(N — 1) 



l R B nm - 



(14) 



In Eqs. (flU]) and (fl"4"]). even for I = 1 the merge conditions 
can be readily satisfied in large graphs for any reasonable 
value of j RB due to the dependences on global graph 
parameters L or N (see also Scc. lVIDl and Appendix A). 
Our APM model merges communities A and B if 



I > 



7 + 1 



(15) 



The merge condition is based only on 7 and the local 
community sizes n and to. For 7 = 1, even small com- 
munities merge with large ones only if there are many in- 
terconnections. The dependence on 7 is consistent with 
the purpose of its introduction in Eq. @ — to allow the 
model to vary the system resolution. 



D. Mitigated resolution limit 

We return to the unweighted system of q cliques in 
Fig. [7] and Sec. IVIBI to show that certain conditions will 
mitigate resolution-limit effects. By design, this circle 
of cliques was constructed to have an unambiguous intu- 
itive answer. Communities are not so clearly defined in 
practice, so we convert the cliques to communities with 
tin edges each, not necessarily maximally connected. We 
also increase the number of intercommunity edges so that 
each community has an average of l ou t edges connected to 
s other communities (qs£ ou t/2 total external edges). The 
original condition for the RBCM for neighboring cliques 
to merge [with U n = m{m — l)/2, l out = 1, and s = 2] is 
given by Eq. ([9]). The new merge condition is 



Irb < 



(16) 



High levels of noise [s ~ O(q) and £ out > 0(1)] tend to 
reduce the effect of the resolution limit because the ratio 
is asymptotic to 7 nB ~ 1. 

For the benchmark in Sec. IV Bl Eq. (TIB")) explains how 
the RBCM can perform very well, despite a resolution 
limit, even when a large number of communities q are 
present (we also subjectively evaluate many values of 
Irb)- On the other hand, more weakly defined communi- 
ties [£i n < m(m — l)/2] tend to increase resolution-limit 
effects, but system noise can substantially and positively 
influence the effects of the resolution limit. 



E. Locality of weighted Potts models 

When considering weighted graphs, the introduction 
of (additional) global dependences should be a "warning 
flag" because global dependences are the source of the 
resolution limit. We show that the APM is remains a 
local model for weighted and directed graphs. 



11 



1. Absolute Potts model 

We generalize results from Sec. I VI Cl for the APM with 
an emphasis on weighted graphs including those with 
weighted missing links. Missing link weights correspond 
to levels of adversarial relations between nodes. "Neu- 
tral" relations use a weight bij = 1 since a weight of 
is an inconsistent community detection model in general. 
The following result also applies to directed graphs. Rep- 
resented as a sum over communities, Eq. becomes 

n s ({a})=Y / {-^s+jn s ) (17) 

s 

where w s and u s are the energy sums of connected and 
missing edges of community s, respectively. For reference 
in Sec. IVI E 31 the unweighted version of Eq. (fTT)) is 

W s (M) = £[-( 7 + l)Z s +7C*] (18) 

s 

where l s is the number of edges and Z™ ax is the maximum 
number of possible edges in community s. 

In Fig.[5Jb), we use Eq. (fTT)) to calculate the condition 
for two arbitrary communities A and B to merge in a 
general graph. A and B are connected by edges with a 
total weight of w and a total missing link weight of u. 
The merge condition is almost trivially given by 

w > ju. (19) 

Note that this merge condition is based only on 7 and 
the connected or missing edges between A and B. The 
APM remains a local measure for general graphs in the 
strongest sense because node assignments are indepen- 
dent of the internal structure of the communities (see 
also Sec. IVI E 4)1 . 



2. Weighted configuration RB Potts model 

A weighted generalization of the RBCM model is 

^cm(M) = E {- w ° + ^*°m) (20) 

where we express it as a sum over all communities. W 
is the total weight of all edges in the system and W s 
is the total weight of all edges in community s (includ- 
ing edges connected to other communities). As with the 
unweighted variant, this weighted model is necessarily 
already a global measure due to W in the sum over W^. 

3. Weighted Erdos-Renyi RB Potts model 

The weighted generalization of the RBER model of 
Eq. ^ increases the global dependence of the model as 



it is proposed in [l~3|. We rewrite the unweighted RBER 
model as a sum over communities 

w)=E(-^wn. (21) 

s 

Equations (fT5|) and (J5TJ show that in the special but im- 
portant case of unweighted graphs, the APM and RBER 
models are coincidentally equivalent if we rescale the null 
model weight by 7 EB = j RB p to explicitly remove the 
global density dependence (see Appendix A). 

We write a conceptual generalization of Eq. (|2Tj) for 
weighted graphs which we use again in Sec. IVI E 41 

= E + lnsP<n- (22) 

s 

Analogous to Z™ ax , wf &x is the "maximum weight sum" 
of community s which must be defined. RB used one 
natural definition of (i) < lax = Wlf ax to obtain [l]| 

n w ER {{a}) = E (-w. + 1 RB PWIT*) (23) 

s 

where W is the average weight over all edges. 

In Fig. [8[b), an arbitrary graph G has three parts: 
two communities A and B, and an arbitrary sub-graph 
G . Communities A and B are connected by a summed 
edge weight w. We ignore the missing link weight sum 
(u = 0) since the model does not account for them. Using 
Eq. (|23[) . the condition for the communities to merge is 

w > r y RB pWnm. (24) 

The dependence on W allows arbitrary changes to in- 
dependent parts of a graph to unintuitively affect each 
other. For example, if we alter the edge weights in sub- 
graph G', we change the average edge weight W. As 
a result, we indirectly change the condition for commu- 
nities A and B to merge even though there are no local 
changes that affect A, B, or the links between them. This 
type of indirect effect caused by global parameters of the 
graph is at the heart of the resolution limit. 



4- Local "Erdos-Renyi" Potts model and "weak" locality 

One can modify the weighted RBER model of Eq. 
to create a local variant. We briefly introduce our own 
variant since the comparison illustrates how "strongly" 
the APM defines a local measure of community structure. 

Another natural interpretation of w" lax in Eq. (f2"2"j) is 
(ii) u)™ ax = w s ^™ ax where w s is the average edge weight 
in the local community s. We also define r y ER — r y RB P 

to 

explicitly remove any dependence on the global density 
of the system. (This was the initial form of the RBER 
model |l3j |. See also Appendix A.) In removing the den- 
sity dependence p, the model is technically no longer an 
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FIG. 9: (Color online) Graph depicts the Zachary karate club 
solved with the APM using weighted edges (relative line 
thickness), 7 = 1, and q = 2 communities by constraint. All 
nodes except 10 are correctly assigned. By our analysis, this 
node appears frequently in both groups (see text). 



"Erdos-Renyi" Potts model; however, in this interpreta- 
tion, Eq. (j2"2")l simplifies to 



Wjsh'(M) = H» + 7 ER I?™)- (25) 



groups because there is no energy difference between the 
two assignments at 7 = 1. (This is a rudimentary iden- 
tification of an overlapping node using a method such as 
in [l3j].) In the actual division, node 10 associated with 
the group depicted by circles. A more complete multires- 
olution analysis in [22j would correctly place node 10. 

We also construct a very large system similar to those 
defined in Sec. IV Bl The system has 40 x 10 6 nodes and 
L = 1 157 634 899 edges assigned in a power-law distri- 
bution of node degrees with an exponent a — — 2. We 
specify the minimum and maximum degrees as fc m i n = 20 
and fcm ax = 500, respectively. The system is randomly 
partitioned into q — 2 443 782 communities in a power- 
law distribution of community sizes with an exponent 
P = — 1 with sizes ranging from n m - m = 10 to n max = 25 
nodes. The average internal community density is set to 
Pin = 0.95. The average graph density is p — 1.45 x 10 -6 . 

We solve the system with 7 = 1/2 in Eq. ^ with the 
algorithm in Sec. Mil using t = 1 trial. (Random com- 
munity edge assignments allow some nodes to be weakly 
connected to their intended communities. Using 7=1/2 
ensures that all but the extreme outliers are properly 
assigned.) The system was solved very accurately with 
V = 1.17 x 10~ 7 in 3.9 hours on a single processor [3lj |. 



This variant uses almost the same energy sum as the 
unweighted RBER model in Eq. (O except that total en- 
ergy of community s is weighted by w s . Equation (|25[) 
is a local model in the sense that only parameters in the 
"neighborhood" of the local communities contribute to 
the energy, but it is a local model in a "weaker" sense 
than the APM because node assignments depend on the 
internal structure (edge weights in this case) of the com- 
munities. 

One can devise applications for such weakly local qual- 
ity functions when influences within a graph need to be 
abstracted for efficiency or due to limited knowledge of 
the full details of the network (e.g., social networks with 
influential personalities). However, despite being a local 
model, using these indirect dependences for community 
assignments (without associated edges between nodes) 
should elicit some caution because similar indirect effects 
on a global level are the source of the resolution limit for 
modularity and the RBPM. 



VII. EXAMPLES 

A common test is the Zachary karate club [EH . It pro- 
vides a small and real example of a social division that 
occurred while the group was under study. The graph 
consists of 34 people with 78 recognized relationships be- 
tween them that are weighted according to the strength 
of the friendships (depicted by the relative line thickness). 
We use the weighted relations in Eq. ([2]) with bij = 1 and 
divide the graph into two parts by constraint as shown in 
Fig. O Our algorithm correctly identifies the communi- 
ties except for node 10 which appears frequently in both 



VIII. CONCLUSION 

We present an exceptionally accurate and local spin- 
glass- type Potts model for community detection: (1) 
our approach employs an absolute energy evaluation as 
opposed to a null model comparison. (2) Its accu- 
racy, even when using a greedy algorithm, is among 
the best of currently available algorithms. (3) The 
model is robust to noise in the system. (4) It is a lo- 
cal measure in the strongest sense of the term for un- 
weighted, weighted (including weighted "adversarial" re- 
lationships), and directed graphs. As such, it corrects a 
resolution-limit pro blem that affects other popular mea- 
sures 

[in nil mm. ( 5 ) Heter ° 

geneous community sizes 
are naturally resolved. (6) The computational demand 
often scales as 0(tL 13 ) where t is the number of op- 
timization trials [generally O(10) or less] and L is the 
number of edges in the system. We have been able to ac- 
curately solve synthetic systems as large as 40 x 10 6 nodes 
and over 10 9 edges [3l[. In Ref. [22j, we illustrated in de- 
tail how this core community detection method may be 
extended to systematically, accurately, and rapidly iden- 
tify general multiresolution structures. 
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Appendix A: RESOLUTION LIMIT AND THE 
ERDOS-RENYI POTTS MODEL 

For unweighted graphs, the RBER model of Eq. (|2T|) . 
based on the Erdos-Rcnyi null model, is not inherently a 
global measure of community structure as is the RBCM, 
based on the configuration null model. The original 
model [lH was defined without the density p where 
7 EJf = j RB p. The ad hoc inclusion of the density carried 
an implicit assumption that j RB is constrained to some 
range, perhaps by ~/ RB ~ 0(1) (otherwise, introducing 
a second constant is not meaningful). It then became a 
Potts model based on an Erdos-Rcnyi null model. 

The justification for including the graph density in the 
model was initially based on heuristic arguments about 
density inequalities that bounded the behavior of 7 BJi . 
Data were also presented using a common, but very 
small, benchmark (discussed in Sec. HVj) that supported 
the approximation of r ) ER oc p. However, the approxima- 
tion is not generally applicable. For example, between 
the systems in Sees. [TV] El and ELD we would need to 
vary j RB by at least 3 orders of magnitude (and arbitrar- 
ily larger if we increase the system size in Sec. IV B[) if we 
wish to consistently identify the most accurate solution 
for each system. If we remove the constant (but graph 
dependent) density p, we trivially remove from Eq. (|21[) 
any dependence on global graph parameters. 

This change is more than a pedantic exercise. Connect- 
ing the RBER model to the system density allowed it to 
automatically scale to solve arbitrary graphs in a semiob- 
jective manner (see Sec. I VI AH . but it also appeared to 
impose a resolution limit [19(. Trivially removing the 
global dependence on p effectively "eliminates" the reso- 
lution limit for the model if one reinterprets the meaning 
of the original model weight r y ER . With this change, we 
assert that there is no genuine resolution limit in the un- 
weighted RBER model as it was originally presented in 
[l3| without the density dependence p. 

The second term in Eq. (|2"Tj) indicates that j ER speci- 
fies the fraction of ^" lax that each community must have 
before it has an energy less than zero. Thus, we reinter- 
pret 7 EH as the minimum edge density of each community 
in a solved partition (or the maximum external edge den- 
sity [HI), but this minimum density is enforced through 
only local constraints. The cost for this freedom is that 
we must choose the "correct" weights j ER for each graph, 
but the best choices are not arbitrary. 

After removing p, we re-analyze the resolution-limit 
results obtained for the RBER model in Sees. IVI Bl and 
IVI CI Using Fig.JSJa), the original condition for two arbi- 
trary unweighted communities A and B to merge is given 
by Eq. (|14[) . Without p, the new merge condition is 

1 > l E R nm ( A1 ) 

which is based only on local variables of communities A 
and B and the independently set 7 ER . 

For the circle of cliques depicted in Fig. the original 
merge condition is given by Eq. (|10[) . The new condition 



for two neighboring cliques to merge is 

1 ER < \- (A2) 

Using the reinterprctation of 7 ER , the value of 7 ER = 1/2 
demands at least a 50% edge density for each community 
to be valid. At m = 3, Eq. (|A2j) demands j ER < 1/9 for 
a merger to occur. Therefore, at j ER = 1/2 the model 
will not experience a resolution limit effect for any global 
scale of N, L, or q for cliques of size m > 3. 

After removing the density and reinterpreting 7 Efl , the 
model is not genuinely subject to a resolution limit be- 
cause the constraints that define the community struc- 
ture are enforced locally. We can then apply concepts 
mentioned in Sec. IVI Al to solve graphs with a local com- 
munity measure. One caveat is that the locality of the 
RBER model does not extend as naturally to weighted 
systems (see Sec. IVI E) . 

Appendix B: VARIATION OF INFORMATION 

We use the variation of information metric [53| for our 
accuracy tests in Sec. |V] The probability that a ran- 
domly selected node from partition A will be a member 
of community k is P{k) = n^/N, where nu is the number 
of nodes in community k and N is the total number of 
nodes in the system. The Shannon entropy is 

k=l 

where qA is the number of communities in partition A. 

The mutual information I(A, B) evaluates the level of 
interdependence in two sets of data. We define a "confu- 
sion matrix" for partitions A and B by identifying how 
many nodes riij of community i of partition A are in 
community j of partition B. The mutual information is 

1A QB / AT\ 

EE < B2 > 

i=l j = l v 1 3 / 

where rii is the number of nodes in community i of par- 
tition A and rij is the number of nodes in community j 
of partition B. The variation of information V(A, B) is 

V(A, B) = H{A) + H(B) - 2I(A, B), (B3) 

which measures the "distance" in information between 
the two partitions A and B. The range of values for VI 
is < V(A, B) < log N. We use base 2 logarithms. 

Appendix C: EXAMPLE NOISE TEST SOLUTION 
WITH THE RBCM 

In Sec. IV Bl we add noise to a strongly defined system 
to test the accuracy of the RBCM of Eq. (J7J compared 
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FIG. 10: (Color online) Plot of VI V vs -y RB for the RBCM. 
In Sec. IV Bl we generate a set of strongly defined communities 
with varying levels of intercommunity noise k ou t- Using the 
greedy algorithm in Sec. Mil we compare the accuracy of so- 
lutions found with the RBCM to our APM. Since the models 
operate differently, we compare the results in Figs. [5] and [6] 
for our APM using 7 = 1 to the best result for the RBCM 
independently determined for each k out . To this end, we in- 
crement 7 RB by 20 steps per decade and calculate VI for each 
solution using the known answer. We then select the best 
7 HB corresponding to the lowest VI average. This example is 
for a system with TV = 512 nodes (see Fig. [4j) with an average 
external degree k ou t ~ 10. See the text regarding other pa- 
rameters defining the distribution of initial node degrees and 
community sizes. Each point is an average over 100 graphs. 



to the APM of Eq. @. A sample system is depicted in 
Fig. [4[ and the accuracy results are summarized in Figs. 
[5] and [5] For the APM, we solve the system with the 
model weight 7 = 1 for all graphs. Figure [TU] shows an 
example of how we select the best result for the RBCM 
as compared to the known answer. 

We start with j RB =0.1 and geometrically increase the 
step size by 10 1 / 20 (i.e., 20 steps per decade of 7 RB ). This 
example is for TV = 512 nodes. The power-law distribu- 
tion exponents arc a — — 2 and /3 = — 1 for the power- 
law degree and the community size distributions, respec- 
tively. Other parameters are: minimum and maximum 
community sizes n nlln = 4 and n max = 50, community 
edge densities pin = 1, average external degree (noise) 
(k) a ~ k ou t — 10, maximum external degree fc ma x = 100, 
and t = 4 trials per solution. Figure [10] shows only one 
best answer for the RBCM. We average over 100 graphs 
for each -f RB , and the best VI average is plotted in Figs. 
[5] and [6] with the result for the APM. 




FIG. 11: (Color online) Plot of VI V vs the average external 
degree k out for the RBCM. In Sec. IV Bl we generate a set 
of strongly defined communities with high levels of intercom- 
munity noise. In Figs. [5] and [(J we vary the initial average 
power-law deg res (k^ a. — kout and solve the networks using 
the greedy algorithm in Sec. IHII and SA where we use a start- 
ing temperature of To = 1.0 x 10~ 4 for the TV = 512 node 
systems. The greedy algorithm initialized into a symmetric 
initial state (qo = TV) outperforms SA in accuracy when us- 
ing the best 7 RB (see Appendix C). In these plots, we further 
examine SA for a range of starting temperatures. Even with 
significantly higher starting temperatures, SA cannot exceed 
the accuracy of the greedy algorithm in this problem. 



The systems are defined with a power-law distribution 
of community sizes and an approximate power-law dis- 
tribution of external noise (see Sec. IV Bp . We solve each 
system using the RBCM with the greedy algorithm in 
Sec. [Till and SA both with t = 4 trials. In Figs. [5] and 
we use starting temperatures of T = 1 x 10~ 4 for 
TV = 512 and T = 1 x 10" 5 for TV = 4096. Note that 
we scale the model energy by the number of edges in 
the system (-1/L) so that it is explicitly equivalent to 
the normalized modularity when 7 RB = 1. SA performs 
slightly worse in accuracy than the greedy algorithm of 
Sec. Mil using a symmetric initial state (qo = TV). 

Given this counter-intuitive result, in Fig. [TT] we ex- 
amine how the accuracy of the SA algorithm is affected 
by the algorithm's starting temperature. We plot the av- 
erage VI V for the best RBCM result (see Appendix C) 
versus the average external degree k out . We test starting 
temperatures spanning 5 orders of magnitude for TV = 
512 nodes. The cooling rate is fixed at Ti+i = 0.999Ti 
where each step i consists of TV randomly proposed state 
changes. For the highest temperatures, the computa- 
tional time dramatically increases due to a significantly 
longer cooling time with no significant improvement in 
accuracy. Thus, a higher starting temperature for SA 
cannot improve its performance sufficiently to match the 
accuracy of the greedy algorithm in this problem. 



Appendix D: NOISE TEST ANALYSIS OF SA AT 
DIFFERENT STARTING TEMPERATURES 

In Sec. lVBl we construct a set of maximally connected 
communities with varied levels of intercommunity noise. 



Appendix E: EVIDENCE OF A PHASE 
TRANSITION IN COMMUNITY DETECTION 

In Fig. [T^l we construct a set of well-defined but noisy 
systems with TV = 4096 nodes from the benchmark in 
Sec. IV Bl An initial random degree distribution is as- 
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FIG. 12: (Color online) Plot of VI V vs the average ex- 
ternal degree k ou t for the APM and RBCM models. Sim- 
ilar to Sec. IV Bl we generate strongly defined communities 
with TV = 4096 nodes and high levels of intercommunity 
noise. We use the greedy algorithm in Sec. IIHI to solve the 
systems for both models. The system is initially assigned 
a random power-law degree distribution with an exponent 
a = — 2, maximum degree fc max = 1200, and average degree 
{k) a (kout — (k) a ). Communities are assigned in a power- 
law distribution with an exponent /3 — — 1, minimum size 
limin = 8, maximum size n max = 24, and density pi n = 1. The 
APM shows sharp accuracy transitions at (i) near k ut — 620 
(no zero energy moves) and at (Hi) near k ou t — 770 (using 
zero energy moves). These roughly correspond to a similar 
transition for the RBCM at (ii). See the text regarding (iv). 
Each point is an average over 25 graphs. 

signed according to a power law with an exponent of 
a = —2, maximum degree fc max = 1200, and average 
degree (k) a . Community sizes are assigned in a power- 
law distribution with an exponent of f3 = — 1. Mini- 



mum and maximum community sizes are n m j n = 8 and 
"max = 24, respectively. We then maximally connect in- 
ternal community edges (density p m = 1). We vary the 
average power-law degree (k) a (the average external de- 
gree k out — (k) a ) and solve the system with the APM 
and RBCM models using the algorithm in Sec. IIIII 

Features (i) and (in) correspond to two related "phase 
transitions" of the system into a "glassy" state for our 
APM. The "complexity" of the energy landscape dramat- 
ically increases near the "critical points" of fc„„ t ~ 630 
(where we disallow zero energy moves) and k ^ ~ 770 
(where we allow zero energy moves), respectively. After 
the transitions, our algorithm in Sec. IIIII is more easily 
trapped in a metastable state when navigating the energy 
landscape. In the intermediate region, the APM can still 
almost perfectly solve the system (in general problems, al- 
lowing zero energy moves does not usually result in such a 
drastic difference in accuracy) . A second aspect of these 
transitions (not depicted) is a generally rapid rise in the 
computational effort required to solve the system which 
peaks near the respective critical points. 

Feature (Hi) at k Q l "l ~ 680 shows that the RBCM dis- 
plays a similar transition. We speculate that the more 
complicated energy landscape of the RBCM actually al- 
lows further optimization as compared to the APM when 
not utilizing zero energy moves in this problem. At fea- 
ture (iv), the best RBCM solution (see Appendix C) ap- 
proaches a trivial partition with q > 3200 communities. 

This community detection transition is similar to tran- 
sitions in the k-SAT (k-SATisfiability) problem found by 
Mezard et al. (54|. The authors showed that the most 
difficult solutions for k-SAT problems are found along 
well-defined loci in the phase diagram of random satis- 
fiability problems. Figure [12] illustrates a similar transi- 
tion in community detection. We will address the phase- 
transition aspects of community detection in more detail 
in an upcoming publication. 
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